A biological neuron is a single nerve cell responsive to stimuli through weighted inputs known as synapses. The weighted stimuli are summed and processed through a particular nonlinearity (sigmoid) associated with the neuron. The output of the neuron may be coupled to the synapses of other neurons forming an interconnection known as a neural network which possesses many desirable properties including the ability to learn and recognize information patterns in a parallel manner.
One conventional technique for processing the input signal vector through an artificial neuron involves forming a dot product combination between the elements of the input signal vector and the weight elements associated with the synapses. The dot product computation typically utilizes a plurality of multipliers coupled one each for receiving an element of the input signal vector and an element of the weight vector. The output signals of the plurality of multipliers are summed and processed through the sigmoid for providing the output signal of the neuron. A more detailed explanation of the dot product operation of conventional neurons is described in the detailed description of the prior art section.
One of the shortcomings of the aforedescribed neuron is the requirement for multiplier circuits which perform the dot product between the input signal vector and the weight vector. In analog form, multiplier circuits tend to be limited in accuracy and sensitive to process variation during manufacturing and temperature variation during operation. On the other hand, four quadrant digital multipliers are often large and complex taking significant portions of the available integrated circuit area, especially for neurons processing large input signal vectors since each element of the input signal vector is typically matched with a dedicated multiplier. Hence, it is desirable to remove multiplier circuits from neural networks as much as possible.
The nonlinear sigmoid function is provided for limiting the magnitude of the output signal should the sum of the products of the input signal vector and the weight elements become excessively large. While it is possible to limit the magnitude of the output signal by normalizing the input signal vector (dividing each element by a predetermined constant), a sigmoid function positioned at the output of the neuron has proven to be a simpler solution. Yet, it is still desirable to eliminate the sigmoid function to simplify the design and reduce the physical size of the neuron thereby allowing more neurons per unit area on the integrated circuit.
Hence, there is a need for an improved artificial neuron for processing an input signal vector through the synapse weight elements without the use of multiplier circuits for performing the dot product combination thereof and further without the sigmoid function for limiting the magnitude of the output signal of the neuron.